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(54) Scene change detector for digital video 

(57) In a method for detecting a scene change 
between a prior video picture and a current video pic- 
ture of a sequence of pictures, an average luminance 
value is determined for a block pair of the prior and cur- 
rent video pictures. Preferably, the blocks of the block 
pair are located, respectively, in the same relative posi- 
tion in the prior and current pictures. An incremental vis- 
ual sensation value is determined using a difference 
between the average luminance values. If the incremen- 
tal visual sensation value exceeds a block contrast 
threshold level, a scene change is indicated. In particu- 
lar, if the minimum of the average luminance values of 



the current and prior picture blocks exceeds a dark 
scene threshold, the incremental visual sensation value 
is determined using the ratio of (a) the absolute value of 
the difference between the average luminance values, 
and (b) the minimum of the average luminance values of 
the current and prior picture blocks. Otherwise, the 
incremental visual sensation value is determined using 
the ratio of (a) the absolute value of the difference, and 
(b) the dark scene threshold. The method may be opti- 
mized by adjusting the block size based on the relative 
amount of motion and the current picture type. 
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^ ,l '4T e " S The valueofthe constant C has been found to be 0.02. which means that, on an scale of 0 to 1 at least 
frfty different lummance levels are required for the contrast between levels to be perceptible by alSman ' 
^ Denotng b 0 = 6.. one can write b s = 6 + So. where 86 is the smallest perceptible luminance change. 



10 which indicates that 
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is proportional to the incremental visual sensation of brightness. 

FIGURE 1 illustrates a comparison between blocks of two consecutive video frames in accordance with th« nr^ 
invention. A current frame, Frame (i), shown at 100, includes a block 1 10 A pr^oT^e ^m^^ P ™ 
.ncl^esa^ 



h height of frame (pixels) 

w width of frame (pixels) 

m height of block (pixels) 

n width of block (pixels) 

55 i frame index 

k horizontal block index (k=1 h/m) 

I- vertical block index (1=1, ;. |W /n) 

X j|k?l pixel intensity of ith frame, kth horizontal block, rth vertical block 
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The threshold T 3 is selected based on the different picture types which are present in the sequence of video 
frames. For example, if there are no B-pictures in the bitstream, e.g., with the sequence I, P, P, .... then T 3 =16 is an 
appropriate choice. If there is one B-picture present, e.g., with the sequence P, B, P, B, .... then T 3 =16 is an appropriate 
choice if the current picture is a B-picture, and T 3 =32 is an appropriate choice if the current picture is a P-picture, and 
5 so forth. 

Next, the block size is adjusted accordingly. An initial (default) block size of 16 x 16 may be used. Then, the block 
size may be adjusted based on v{x][y]. For example, if v[x][y]=1 , then the block size may be increased, e.g., to 16 x 32 
or 32 x 32. Similarly, rf v[x][y]=0 • tnen tne block size may be decreased, e.g., to 8 x 16. However, note that the block 
size should not be increased such that the block crosses over the right and/or bottom boundary of a macroblock. Gen- 
w erally, the block size should be larger when the motion is faster. Moreover, the largest allowed block size may be limited 
in some applications to 32 x 48 pixels. 

Next, the average luminance of each block in a frame (or top field) is determined. For the (i)th frame, block (k, I), the 
average luminance is: 



15 ^ m-1 n-1 

\kh+d,lw+c2» 
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d=Oc2=0 



20 for k=1, ... h/m, and 1=1 w/n. c1 and c2 are dummy counting indexes. Next, the Wock-luminance-increment 8B ikl 

between the (i)th and (i-1) frames (or top fields) is determined by: 



8 e /,*,/ =e W S M,v 



25 Furthermore, the relative block-incremental-contrast ?>C l k , for the (i)th frame, block (k.l). can be defined by: 

\ SB iJcJ 



SC iJcJ = ' 



—r- 1. \Jorwm{B itk j,B i _ VcJ }> T 0 ; 



T 0 is a threshold which indicates that a scene is considered to be a dark scene. Generally, T 0 =25.5 may be used, which 
is 1 0% of the maximum gray scale level 255. 

For a scene change, a significant threshold T 1 of the relative change of block luminance contrast is set as: 
7" 1 =15~25C. Now. consider an index array, index[k][l], for k=1 h/m, 1=1 w/n, defined by: 



f I, if SC:j, , > T, 

index[k]D] = i A rlt • 

[0, Otherwise. 



Then, if approximately 80-90% of the blocks in a frame have a relative block-incremental-contrast which is greater than 
the significant threshold, i.e., 

IhlmkwInS 
X Xindex[k][l] 



then, in accordance with the present invention, a scene change is indicated. The range of 80-90% was determined 
based on extensive testing, but the actual optimal figure may vary with the particular scene. Note that the mathematical 
expression x J denotes rounding of the non-integer x to the next lowest integer. 
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FIGURE 2 is a block diagram of a scene change detector in accordance with the present invention The detector 
shown generally at 200. includes input terminals 205 and 210. At input terminal 205, pixel data from the current frame' 
X ifk| i, Is received and provided to a block average function 215 to produce the average luminance value for each block 
in the ith frame, B tK] . Similarly, at input terminal 210, pixel data from the previous frame. X M k , is received and provided 
to a block average function 220 to produce the average luminance value for each block in the (i-1 )th frame. B i _ 1 k ( . 

Minimizer unit (MIN) 225 determines min{B itk l , B M and outputs this term to a divisor function 23o/Meanwhile 
subtracter 235 determines 6 B i k i = B iiKr B^ tKh Absolute value function 240 determines |5 B iKt \, and provides this term 
to the divisor 230. The divisor 230 determines the relative block-incremental-contrast 8C, J, for the (Oth frame block 
(Kl). depending on whether rninfe^. B M ^>T a bC iAI is then provided to a threshold function 235 which determines 
whether block (k.1) is indicative of a scene change (e.g. t whether 3C fk f>T v If so. an index[k][l] may be set accordingly 
Accumulator 240 accumulates the scene change result for each block, and sums the result over the entire frame or a 
portion thereof . Finally, threshold function 250 receives the summed result from accumulator 240, and uses the thresh- 
old T 2 to determine whether a scene change for the overall frame is indicated. 

The scene change detection system of the present invention was tested extensively using different video 
sequences. In particular, the "Football". "Mobile Calendar". "Flower Garden", and "Table Tennis" video sequences 
described in Test Model Editing Committee. "Test Model 5", ISO/I EC JTC 1 /SC29/WG 1 1 MPEG93/457 April 1993 were 
analyzed, along with the "Destruct". "Street Organ", "Silent", and "Fun Fair" video sequences, described in the M hoc 
group on MPEG-4 video VM editing, "MPEG-4 Video Verification Model Version 3.0". ISO/IEC JTC1/SC29/WG11 
N1277. Tampere, Finland, July 1996. 

Sample test results of the scene detection system of the present invention are shown in Table 1 . below The thresh- 
olds were selected as T 1= 0.3 and T 2 =0.85, and the block size was m=l6, n=32. The particular video sequence is iden- 
tified in the first column. The sequence of frames involved is indicated in the second column. For example [0 50] 
indicates that frames 0 through 50 were analyzed for a scene change. The third column indicates whether a scene 
change was detected, and if so. in which frames. For example, a scene change in a third frame means that a scene 
change between the second and third frames was detected. The fourth column, if applicable, provides additional infor- 
mation on the nature of the video sequence. 
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Sequences 


Frames 


Scene Change 


Comments 


Football 


[0:50] 


No 


Fast motion 


Mobile Calendar 


[0:44] 


No 




Street Organ 


[0:50] 


No 




Silent 


[0:50] 


No 


No motion to motion 


Flower Garden 


[0:30] 


No 


Camera panning 


Fun Fair 


[0:50] 


No 


Fast motion 


Table Tennis 


[90:100] 


97th 




Destruct 


[0:40] 


25th.26th 


A bright light 


Combination of any two sequences 




Yes 


scene change detected every time 



Moreover, for frames in the above video sequences in which a scene change was detected, coding efficiency was 
examined using the MPEG-2 WG-1 1 programs. Coding efficiency is measured by determining the number of bits gen- 
erated by the coding method to achieve a given image quality. Specifically, for a constant quality level, fewer coding bits 
are indicative of higher coding efficiency. It was determined that, if a frame with a scene change detected in accordance 
with the present invention is coded as a P-picture type, then more than 46% of the macroblocks are coded as l-pictures. 
Thus, the scene detection system of the present invention was found to operate as expected since it successfully 
located frames which are difficult to efficiently code using predictive coding. Advantageously, such frames can be coded 
quality*" 68 "** ^"^^ °* enCOder allocates more bits for '-pictures, thereby also improving image 

Generally, in a typical encoder, such as an MPEG-2 encoder using the Test Model 5 standard, there is a process 
for selecting a macroblock coding type for each macroblock of a P-picture or B-picture. Such a process will determine 
whether the macroblock should be coded as an intra-coded block (l-coded) or non-intra-coded block (P- or B-coded) 
based on which coding type provides better quality or uses fewer data bits. For a P-picture or B-picture. if the prediction 
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is efficient, only a small proportion of macroblocks in a picture will be l-coded (e.g., less than five per cent). This is desir- 
able as l-coded blocks consume a relatively large number of data bits since there is no temporal compression. 

If the proportion of l-coded macroblocks in a picture is greater than, e.g., thirty or forty per cent, then the picture 
quality will be poor. In this case, prediction coding is inefficient for the picture, as would be expected at a scene change. 
5 Thus, when a scene change occurs, it is generally desirable that the first frame of the new scene should not be coded 
as a P-picture. 

Although the invention has been described in connection with various specific embodiments, those skilled in the art 
will appreciate that numerous adaptations and modifications may be made thereto without departing from the spirit and 
scope of the invention as set forth in the claims. For example, the various threshold levels set forth herein may be 

10 adjusted according to the particular scene or video sequence which is analyzed. That is. some types of video 
sequences, such as action movies, may be characterized by more frequent and pronounced scene change activity. 
Moreover, specific lighting conditions may be associated with a particular video sequence, e.g., such as a horror film, 
where lighting levels may be relatively low throughout the sequence. In this case, the scene change detection thresh- 
olds can be adjusted accordingly. 

is Moreover, it may be desirable to analyze only a portion of a video picture to determine a scene change, or different 
portions may be analyzed using different thresholds. For instance, in a video sequence of a landscape scene with a rel- 
atively dark earth at the bottom part of the picture and a relatively bright sky at the top part of the picture, a more sen- 
sitive scene change threshold may be used for the bottom part of the picture. Similarly, different sized blocks may be 
used in different regions of a picture. For instance, when motion is more prevalent toward the middle of a picture than 

20 toward the edges, larger block sizes may be used in the middle of the picture. 

Claims 

1 . A method for detecting a scene change between a prior video picture and a current video picture, comprising the 
25 steps of: 

determining average luminance values of a block pair of said prior and current video pictures; and 
determining an incremental visual sensation value using a difference between said average luminance values; 
wherein: 

30 if said incremental visual sensation value exceeds a block contrast threshold level, a scene change is indi- 

cated. 

2. The method of claim 1 , wherein said block contrast threshold level is approximately fifteen to approximately twenty- 
five times a Weber fraction constant. 

35 

3. The method of claim 1 or 2, wherein said blocks of said block pair are located, respectively, in the same relative 
position in said prior and current pictures. 

4. The method of one of the preceding claims, comprising the further step of: 

40 

determining a minimum of said average luminance values of said current and prior picture blocks, wherein: 

if said minimum exceeds a dark scene threshold, said incremental visual sensation value is determined 
using the ratio of (a) the absolute value of said difference, and (b) said minimum; 
45 else, said incremental visual sensation value is determined using the ratio of (a) the absolute value of said 

difference, and (b) said dark scene threshold. 

5. The method of claim 4, wherein: 

so said dark scene threshold is approximately 10% of a maximum gray level. 

6. The method of one of the preceding claims, wherein: 

said difference between average luminance values is determined for a plurality of block pairs of said prior and 
55 current video pictures; and 

said incremental visual sensation value is determined for each of said block pairs using said differences; 
wherein: 
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if said incremental visual sensation value exceeds the block contrast threshold level for a threshold propor- 
tion of block pairs in said current and prior video pictures, a scene change is indicated. 

7. The method of claim 6, wherein said threshold proportion is approximately 80% to approximately 90%. 

5 

8. The method of one of the preceding claims, comprising the further step of: 

determining a relative amount of motion between said blocks of said block pair; and 
adjusting a size of said blocks based on said relative amount of motion. 

w 

9. The method of claim 8, wherein the size of said blocks is increased as said relative amount of motion increases. 

1 0. The method of claim 8, wherein said step of determining a relative amount of motion comprises the further steps of: 

15 determining a sum of the absolute value of a horizontal motion vector and the absolute value of a vertical 

motion vector; 

wherein said horizontal and vertical motion vectors are indicative of horizontal and vertical motion, 
respectively, of a video image of said current picture block relative to a video image of said prior picture block; 
and 

20 determining if said sum exceeds a motion threshold. 

11. The method of claim 10 f wherein: 

said motion threshold is adjusted according to a picture type of said current picture. 

25 

1 2. An apparatus for detecting a scene change between a prior video picture and a current video picture, comprising; 

means for determining average luminance values of a block pair of said prior and current video pictures; and 
means for determining an incremental visual sensation value using a difference between said average lurrri- 
30 nance values; wherein: 

if said incremental visual sensation value exceeds a block contrast threshold level, a scene change is indi- 
cated. 

The apparatus of claim 12, wherein said block contrast threshold level is approximately fifteen to approximately 
twenty-five times a Weber fraction constant. 

The apparatus of claim 12 or 13, further comprising: 

means for determining a minimum of said average luminance values of said current and prior picture blocks, 
wherein: 



35 13. 

14. 

40 



if said minimum exceeds a dark scene threshold, said incremental visual sensation value is determined 
using the ratio of (a) the absolute value of said difference, and (b) said minimum; 
else, said incremental visual sensation value is determined using the ratio of (a) the absolute value of said 
difference, and (b) said dark scene threshold. 

1 5. The apparatus of one of claims 1 2 to 1 4, further comprising: 

means for determining said difference between average luminance values for a plurality of block pairs of said 
prior and current video pictures; and 

means for determining said incremental visual sensation value for each of said block pairs using said differ- 
ences; wherein: 

if said incremental visual sensation value exceeds the block contrast threshold level for a threshold propor- 
tion of block pairs in said current and prior video pictures, a scene change is indicated; 

16. The apparatus of claim 15, wherein said threshold proportion is approximately 80% to approximately 90%. 
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1 7. The apparatus of one of claims 1 2 to 1 6, further comprising: 

means for determining a relative amount of motion between said blocks of said block pair; and 
means for adjusting a size of said blocks based on said relative amount of motion. 

5 

18. The apparatus of claim 17, further comprising: 

means for increasing the size of said blocks as said relative amount of motion increases. 

10 19. The apparatus of claim 17. wherein said means for determining a relative amount of motion further comprises: 

means for determining a sum of the absolute value of a horizontal motion vector and the absolute value of a 
vertical motion vector; 

wherein said horizontal and vertical motion vectors are indicative of horizontal and vertical motion, 
is respectively/of a video image of said current picture block relative to a video image of said prior picture block; 

and 

means for determining if said sum exceeds a motion threshold. 
20. The apparatus of claim 19, further comprising: 

20 

means for adjusting said motion threshold according to a picture type of said current picture. 
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(54) Scene change detector for digital video 

(57) In a method for detecting a scene change 
between a prior video picture and a current video pic- 
ture of a sequence of pictures, an average luminance 
value is determined for a block pair of the prior and cur- 
rent video pictures. Preferably, the blocks of the block 
pair are located, respectively, in the same relative posi- 
tion in the prior and current pictures. An incremental vis- 
ual sensation value is determined using a difference 
between the average luminance values. If the incremen- 
tal visual sensation value exceeds a block contrast 
threshold level, a scene change is indicated. In particu- 
lar, if the minimum of the average luminance values of 



block (w) 




150 



the current and prior picture blocks exceeds a dark 
scene threshold, the incremental visual sensation value 
is determined using the ratio of (a) the absolute value of 
the difference between the average luminance values, 
and (b) the minimum of the average luminance values of 
the current and prior picture blocks. Otherwise, the 
incremental visual sensation value is determined using 
the ratio of (a) the absolute value of the difference, and 
(b) the dark scene threshold. The method may be opti- 
mized by adjusting the block size based on the relative 
amount of motion and the current picture type. 
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